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The  Honorable  Frank  C.  Carlueei 
The  Secretary  of  Defense 


Dear  Mr.  Secretary: 


We  testified  at  an  October  1987  hearing  before  the  Subcommittee  on 
Legislation  and  National  Security.  House  Committee  on  Government 
Operations  on  the  Department  of  Defense  ( txu> )  program  for  acquiring 
threat  simulators.  fSce  app.  I.)  These  simulators  are  intended  to  imitate 
Soviet  air  defense  weapon  systems,  including  surface-to-air  missiles  and 
guns. 

Our  testimony  focused  on  the 

need  to  prevent  the  services  from  paying  to  develop  simulators  for  a 
particular  threat  more  than  once  and 

acquisition  of  simulators  which  misrepresent  threat  systems  and  thus 
adversely  affect  testing  of  major  I'.S.  weapon  systems  and  training  of 
C.S.  forces. 


Rather  than  developing  a  simulator  for  a  particular  threat  once  and  pro¬ 
ducing  it  in  sufficient  quantities  to  meet  their  needs,  the  Army.  Air 
Force,  and  Navy  are  paying  contractors  for  multiple  developments  of 
simulators  for  the  same  threat.  Substantial  costs  could  be  avoided  by 
strengthening  ix>i>  controls  to  preclude  the  multiple  developments. 

The  multiple  developments  are  occurring  in  simulator  programs  for  so- 
called  emit  ter- receiver-processors  (kki*)1  and  emitters. 


We  found  in  Kite  programs,  for  example,  that  since  1980  the  Army  and 
Navy  each  has  acquired  a  simulator  for  the  Soviet  SA-0  air  defense  sys¬ 
tem  while  the  Air  Force  has  acquired  two.  The  cost  of  these  four  simula¬ 
tors  was  $31.9  million.  Two  SA-0  simulators  had  already  been  developed 
before  1980.  Thus,  the  services  have  paid  for  the  development  of  six 
different  SA-0  simulators.  The  total  cost  associated  with  the  multiple 


T.KI’S  ineor|>oratc  all  elements  of  a  radar  system,  including  the  emitter,  rereiver,  and  ass, mated  „tc 
nal  processor  and  thus  are  intended  to  represent  the  total  funetional  eapnbility  of  radars  asso,  luted 
with  Soviet  surface-to-air  missile  and  nun  systems 

-Knutlers  provide  only  electronic  emissions  or  signals  intended  to  match  those  of  threat  rud.it  - 
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erp  developments  completed  or  planned  since  January  1980  exceeded 
$560  million. 

Similarly,  in  emitter  programs,  we  found  13  different  simulators  for  the 
SA-6,  10  simulators  for  another  threat,  12  for  another,  and  so  on.  In 
total,  we  found  that  two  or  more  simulators  were  acquired  or  planned 
since  January  1980  for  23  threat  systems  at  a  cost  exceeding  $600  mil¬ 
lion.  The  multiple  developments  are  occurring  mainly  within  the  Air 
Force. 

The  services  believe  that  their  separate  developments  are  justified 
because  the  simulators  are  of  different  designs  and  were  acquired  at  dif¬ 
ferent  times  to  meet  their  unique  requirements.  While  different  designs 
may  sometimes  be  warranted  to  meet  unique  requirements,  we  believe 
that  the  differences  could  be  accommodated  by  varying  the  designs 
around  the  same  basic  simulator  components,  for  example,  the  antenna 
and  transmitter.  Alterations,  such  as  making  the  simulator  mobile  or  not 
mobile,  do  not,  in  our  opinion,  justify  separate  development  efforts.  We 
also  believe  that  with  proper  coordination  and  planning,  the  services 
should  be  able  to  predict  their  collective  needs  so  that  they  could  be  met 
through  a  single  development  program. 

We  believe  that  the  multiple  erp  developments  were  caused  by  (1)  a  per¬ 
ceived  lack  of  authority  within  dod  for  managing  the  services’  simulator 
programs  and  (2)  a  desire  by  the  services  to  manage  and  control  their 
separate  programs  and  the  attendant  resistance  to  joint-sendee  efforts. 

The  Joint  Executive  Committee  on  Air  Defense  Threat  Simulators,  called 
EXCOM,  and  its  agent  committee  called  Crossbow-S,  have  attempted  to 
encourage  cooperation  among  the  services/  But,  these  efforts  have  not 
yet  resulted  in  an  effective  joint  program.  New  EXCOM  and  Crossbow-S 
charters  have  been  proposed  to  mo’-e  specifically  define  their  roles  and 
responsibilities.  The  proposed  EXCOM  charter  clearly  identified  the 
EXCOM  as  the  dod  decision  authority  for  simulator  programs.  But,  the 
services  strongly  opposed  the  new  charters  because  of  the  authority 
which  would  have  been  vested  in  the  Committees,  and  the  issue  remains 
unresolved. Regarding  the  emitter  programs,  we  found  an  apparent  man¬ 
agement  void  at  the  dod  level.  EXCOM  and  Crossbow-S  representatives 
said  that  they  had  concentrated  management  efforts  on  the  major  erp 


3 DOD  established  the  EXCOM  in  1983  to  manage  its  simulator  program.  EXCOM  is  supposed  to 
review  and  approve  simulator  programs  and  related  resource  commitments.  Crossbow-S  is  supposed 
to  accomplish  the  coordination  of  simulator  programs  as  EXCOM's  agent 
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Simulators 
Misrepresent  Threats 


programs  and  were  not  adequately  staffed  to  oversee  emitter  programs 
In  view  of  the  widespread  multiple  developments  in  emitter  programs, 
especially  in  the  Air  Force,  control  over  the  requirements  definition  pro¬ 
cess  is  needed  to  preclude  repetitive  development  of  simulators  for  the 
same  threats. 

Duplication  in  simulator  programs  may  further  proliferate  if  planned 
acquisitions  of  simulators  by  the  Director  of  Operational  Test  and  Kval 
nation  are  not  carefully  controlled. 


Simulators  which  significantly  misrepresent  threat  systems  are  being 
used  to  test  the  effectiveness  of  1  \S.  weapons  and  to  train  pilots  in  how 
to  combat  hostile  systems.  The  potential  consequences  in  combat  could 
be  serious. 

Of  4(>  different  simulators  examined.1  we  found  that  35  deviated  sub¬ 
stantially  from  intelligence  estimates  of  the  threat  characteristics.  The 
simulators'  deviations  involved  technical  characteristics  of  the  associ¬ 
ated  radars  which  affect  the  system's  range,  accuracy,  and  resistance  to 
countermeasures,  and  thus  the  overall  effectiveness  of  the  air  defense 
system. 

The  faulty  simulators  are  being  used  to  test  important  systems,  such  as 
the  ALQ-ltil  jammer  which  provides  self-protection  for  the  B- 1 B 
bomber.  They  are  also  being  used  in  training  such  as  Green  Flag,  the  Air 
Force's  main  electronic  combat  training  exercise. 

The  results  of  testing  lT.S.  systems  and  training  pilots  could  be  different 
if  the  simulators  used  more  closely  resembled  the  threat  systems 

The  faulty  simulators  are  being  acquired,  at  least  in  part,  because  of 
weak  internal  controls  within  the  services.  The  Army  and  Navy  do  not 
have  a  sufficient  separation  of  responsibilities  for  the  acquisition,  test¬ 
ing.  and  acceptance  of  simulators.  Decisions  to  accept  simulators  with 
known  deficiencies  were  made  by  officials  responsible  for  their  develop¬ 
ment  without  any  required  review  by  or  approval  from  potential  users. 
A  similar  situation  existed  in  the  Air  Force. 


'The  pi  simulal or  s\  sinus  included  ! huso  fielded  since  .lanuar\  l!*Se  ami  these  in  development  In: 
which  characteristics  had  hern  determined 
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We  also  noted  that  ixto  was  not  overseeing  the  acquisition  process  to 
assure  adequate  simulator  quality. 


Recommendations 


We  recommend  that^the  Secretary  of  Defense 

•  assure  that  the  EXCOM  and  Crossbow-S  or  other  appropriate  ixtu  ele¬ 
ments  execute  responsibility  and  authority  for  centrally  managing  simu¬ 
lator  programs  to  provide  for  timely  identification  and  consolidation  of 
simulator  requirements  and  for  disapproval  of  programs  representing 
unwarranted  development; 

.>•  require  the  sendees  to  strengthen  internal  controls  over  simulator  acqui¬ 
sitions  by  segregating  responsibilities  for  development,  testing,  and 
acceptance  of  simulators  as  valid  representations  of  the  threat;  and 

•  assign  to  an  appropriate  dod  element  the  responsibility  for  monitoring 
the  quality  of  simulators  acquired  and  participating  in  the  acquisition 
process  as  necessary  to  assure  the  adequacy  of  simulators.  — 


Objective,  Scope,  and 
Methodology 


The  objective  of  our  review  was  to  examine  whether  dod  was  effectively 
controlling  and  coordinating  its  threat  simulator  program  to  (l)  prevent 
duplication  in  acquisitions  by  the  military  services  and  (2)  assure  that 
simulators  acquired  were  adequate  for  their  intended  use. 


We  focused  on  Army,  Air  Force,  and  Navy  programs  for  air  defense 
threat  simulators  completing  development  since  January  1980  and  those 
planned  for  the  future.  In  evaluating  these  programs,  we  also  considered 
those  simulators  that  were  already  available.  We  examined  program 
acquisition  plans,  requirements  documents,  intelligence  estimates  of 
threat  capabilities,  test  and  evaluation  reports,  and  other  records  bear¬ 
ing  on  our  objectives.  We  discussed  the  duplication  in  simulator  develop¬ 
ments,  the  adequacy  of  simulators  in  representing  the  threat,  and  other 
matters  with  dod  and  service  representatives  responsible  for  managing 
the  simulator  programs.  We  did  not  request  official  dod  comments  on 
this  report.  Our  review  was  conducted  in  accordance  with  generally 
accepted  government  auditing  standards  from  June  1986  to  August 
1987. 


The  head  of  a  federal  agency  is  required  by  31  1. S.C. 720  to  submit  a 
written  statement  on  actions  taken  on  our  recommendations  to  the  Sen¬ 
ate  Committee  on  Governmental  Affairs  and  the  House  Committee  on 
Government  Operations  not  later  than  00  days  after  the  date  of  the 
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report  and  to  the  House  and  Senate  Committees  on  Appropriations  with 
the  agency’s  first  request  for  appropriations  made  more  than  fit)  days 
after  the  date  of  the  report. 

We  are  sending  copies  of  this  report  to  the  Chairmen  of  the  above  Com¬ 
mittees  and  the  House  and  Senate  Committees  on  Armed  Services.  Cop¬ 
ies  are  also  being  sent  to  the  Director,  Office  of  Management  and  Budget 

Sincerely  yours. 


Frank C.  Conahan 
Assistant  Comptroller  General 
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L'nited  States  General  Accounting  Office 
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Delivery 

Expected  at  10:00 
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October  21,  1987 


Statement  of 
Richard  Davis 
Associate  Director 

National  Security  and  International  Affairs  Division 
Before  the 

Legislation  and  National  Security  Subcomnittee 
of  the 

Contnittee  on  Government  Operations 
l'nited  States  House  of  Representatives 


GAO/T-NSIAD-88-3 


Page  6 


GAO  NSIAD-88-93  Electronic  Warfare 


Appendix  I 

Testimony  on  the  Department  of  Defense 
Threat  Simulator  Program 


Mr.  Chairman  and  Members  of  the  Subcommittee: 

I  am  pleased  to  appear  before  you  today  to  discuss  our  review  of 
the  Department  of  Defense  (DOD)  threat  simulator  program.  In  a 
June  5,  1986,  request,  the  Chairman  asked  us  to  examine  whether  DCD 
is  effectively  controlling  and  coordinating  this  program  to  (1) 
prevent  duplication  in  the  acquisition  of  simulators  by  the 
military  services  and  (2)  assure  that  simulators  acquired  are 
adequate  for  their  intended  use.  Our  review  focused  on  those 
simulators  intended  to  imitate  Soviet  air  defense  weapon  systems, 
including  surface-to-air  missiles  (SAMs)  and  guns. 

BACKGROUND 

Soviet  air  defense  systems  pose  a  potential  threat  to  the  aviation 
elements  of  the  Air  Force,  Navy,  and  Army.  These  Soviet  systems 
include  numerous  land-based  SAMs,  referred  to  as  the  SA-3,  SA-4 , 
SA-6,  and  so  on.  They  also  include  naval  variants  of  land-based 
SAM3 ,  such  as  the  SA-N-6  and  SA-N-7,  as  well  as  assorted  gun 
systems.  Many  of  the  systems  have  radars  which  are  used  to  detect 
and  track  target  aircraft  and,  in  some  cases,  to  guide  the  missile 
to  the  target  or  direct  gunfire. 

As  a  part  of  the  effort  to  contend  'with  this  potential  threat,  the 
military  services  acquire  simulators  of  the  Soviet  weapons.  One 
important  use  of  these  simulators  is  to  test  the  effectiveness  of 
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our  electronic  warfare  equipment  developed  to  counter  the  Soviet 
systems.  For  example,  the  capability  of  our  aircraft  self¬ 
protection  jammers  to  deceive  or  otherwise  interfere  with  Soviet 
radars  is  tested  against  the  simulators.  Simulators  are  also  used 
to  test  the  capability  of  aircraft  radar  warning  receivers  ( RWRs ) 
to  alert  U.S.  pilots  that  they  have  been  detected  by  a  Soviet 
radar-controlled  weapon.  Finally,  simulators  are  used  to  train 
pilots  in  an  environment  that  resembles  realistic  combat 
conditions . 

Simulators  acquired  by  the  services  are  classed  on  the  degree  to 
which  they  duplicate  the  threat  systems.  For  example: 

--  Some  simply  provide  electronic  emissions  or  signals  intended  to 
match  those  of  threat  radars  and  are  commonly  called  emitters. 

--  Others  incorporate  all  elements  of  a  radar  system,  including  the 
emitter,  receiver,  and  associated  signal  processor  and  thus  are 
intended  to  represent  the  total  functional  capability  of  a 
threat  radar.  These  are  called  emitter-receiver-processors 
( ERPs )  . 

--  Still  other  simulators  are  not  only  functionally  representative 
of  the  threat,  but  are  built  tc  look  like  the  actual  threat 
system. 
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THE  SERVICES  ARE  PAYING  FOR 
THE  DEVELOPMENT  OF  SIMULATORS 
MORE  THAN  ONCE 

Mr.  Chairman,  our  review  showed  that  DOD  needs  to  strengthen  its 
controls  over  the  services’  simulator  programs  to  prevent  the 
services  from  paying  contractors  to  develop  simulators  for  a 
particular  threat  more  than  once.  Substantial  costs  could  be 
avoided  if  simulators  were  designed  and  developed  only  once  and 
then  produced  in  sufficient  quantities  to  meet  the  needs  of  all 
services.  DOD  has  taken  some  measures  to  encourage  cooperative 
efforts  by  the  services;  however,  these  measures  have  not  been 
fully  effective. 

Table  1  shows  the  number  of  times  the  services  paid  contractors  tc 
develop  simulator  systems  for  the  same  threat. 
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Title  Is  Number  of  Tines  Services  Developed  ERP  Simulators 


Threat 

ERPs  developed  since 
1/80  or  planned 

system 

Armv 

Air 

Force 

Navy 

SA-3  t 

SA-N-1 

• 

_ 

3 

SA-4 

l 

1 

- 

SA-6 

1 

2 

1 

SA-8 

SA-N-4 

1 

2 

1 

SA- 1 C / 
SA-N-6 

- 

1 

2 

SA-11  ' 
SA-N-7 

1 1 

- 

2 

SA- 1  5 

SA-N-9 

i 

- 

1 

BARLOCK*5 

- 

1 

1 

FLAT  FACE*5 

1 

- 

T 

KITE  SCREECH*5  - 

- 

2 

LONG  TRACK*5 

1 

2 

- 

THIN  SKIN*5 

1 

- 

1 

TCP  STEER*5 

- 

- 

2 

Total 

Available  Total 

as  of  1/80  simulators  Cost3 

(millions) 

4  7  $40.5 

1  3  14.0 

2  6  31.9 

4  27.8 

3  182.0 

3  47.2 

2  97.2 

1  3  11.5 

2  4  13.8 

2  5.5 

1  4  34.0 

1  3  10.1 

2  45 . 6 

ssjiq 


aThe  costs  shewn  pertain  to  these  simulators  completing  development 
since  January  198C  or  planned  for  the  future.  Costs  of  simulators 
already  available  as  cf  January  1980  were  not  determined. 

bThe  North  Atlantic  Treaty  Organization  (NATO)  code  names  for  radars 
used  with  Soviet  air  defense  systems. 
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Our  review  was  focused  on  simulators  acquired  since  January  1980 
and  those  to  be  acquired.  However,  in  evaluating  these 
acquisitions  and  to  gain  more  insight  into  the  extent  of 
duplication,  we  also  considered  those  simulators  that  were 
already  available.  For  example,  since  January  1980,  the  Army  and 
Navy  each  has  acquired  a  simulator  for  the  SA-6  while  the  Air 
Force  has  acquired  two,  costing  $31.9  million.  Meanwhile,  two 
SA-6  simulators  had  already  been  developed.  Therefore,  the 
services  paid  to  develop  six  different  SA-6  simulators. 


i 

I 

i 


I 

l 


Table  2  shows  examples  of  emitter  simulators  developed  more  than 
once.  The  multiple  developments  are  occurring  almost  exclusively 
within  the  Air  Force. 


Table  2:  Number  of  Times  Services  Developed  Emitter  Simulators 


Threat 

system 

SA-2 

SA-3  / 

SA-N-1 

SA-6 

SA-8/SA-N-4 
FIRE  CAN3 
FLAP  WHEEL3 
GUN  DISH3 
LONG  TRACK3 
MUFF  COBB3 
OWL  SCREECH3 


Emitters  acquired 
since  1/80  or  planned 
Army  Air  Force  Navy 


5 

6 

8  1 

3 

5 

6 
3 

2  1 

2  1 


Available 
as  of  1/80 


3 


Tot  a . 

s  inul atcrs 


1  6 


aNAT0  code  names  for  radars  used  with  Soviet  air  defense  systems. 
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We  found  that  two  or  more  emitter  simulators  were  acquired  or 
planned  since  January  1980  for  23  threat  systems.  We  could  not 
associate  a  simulator  cost  to  individual  threat  systems  because 
the  simulators  can  represent  more  than  one  threat,  and  the 
services  did  not  keep  records  on  the  simulator  cost  applicable  to 
each  threat.  However,  the  cost  associated  with  these  emitter 
programs  since  January  1980  was  about  $602.4  million. 

Mr.  Chairman,  I  would  like  to  point  out  that  the  simulators 
differ  in  appearance  and  other  characteristics.  In  particular, 
the  Army's  are  designed  to  look  like  Soviet  systems  whereas  the 
Air  Force's  and  Navy's  are  not.  However,  the  simulators  were 
independently  designed  and  developed  to  imitate  the  same  threat 
systems . 

This  situation  is  similar  to  that  found  during  our  recent  review 
of  Air  Force  and  Navy  RWRs  on  which  we  testified  before  this 
Subcommittee  earlier  this  year.  Our  work  on  RWRs  showed  that  the 
Air  Force  and  Navy  were  acquiring  several  different  RWRs  to 
accomplish  the  same  function  against  a  common  threat. 

The  services  believe  that  their  separate  developments  are 
justified  because  the  simulators  are  of  different  designs  and 
were  acquired  at  different  times  to  meet  their  unique 
requirements.  We  disagree  that  separate  developments  were 
justified.  While  we  do  not  dispute  that  different  designs  may 
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f 


sometimes  be  warranted  to  meet  unique  requirements,  we  believe 
that  the  differences  could  be  accommodated  by  varying  the  designs 
I  around  the  same  basic  simulator  components.  The  components  of  a 

threat  radar  simulator,  such  as  the  antenna  and  transmitter, 
should  be  the  same.  Alterations,  such  as  making  the  simulator 
mobile  or  not  mobile  or  making  it  look  like  the  threat,  does  not, 
in  our  opinion,  justify  separate  development  efforts. 

We  also  disagree  that  the  different  timing  of  service 
requirements  justifies  separate  development  efforts.  Obviously, 
threat  systems  are  deployed  at  discrete  points  in  time.  With 
proper  coordination  and  planning,  the  services  should  be  able  to 
predict  their  collective  needs  such  that  they  could  be  met 
through  a  single  development  program.  We  believe  that 
substantial  costs  could  be  avoided  if  simulators  were  designed 
and  developed  only  once  and  then  produced  in  sufficient 
j  quantities  to  meet  the  needs  of  all  services. 

EFFECTIVE  POD  OVERSIGHT  OF 

SIMULATOR  DEVELOPMENTS  IS  MISSING 

I  - 

i 

Regarding  the  major  simulator  programs  involving  ERPs,  we  believe 
the  multiple  developments  of  simulators  stem  from  a  combination 
of  two  factors:  (1)  a  perceived  lack  of  authority  within  DOT  for 

j  managing  the  services'  simulator  programs  and  (2)  a  desire  by  the 

services  to  manage  and  control  their  separate  programs  and  the 
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attendant  resistance  to  joint-service  efforts.  Regarding  the 
emitter  programs,  we  found  an  apparent  management  void  at  the  DOD 
level  and  a  need  to  improve  the  Air  Force’s  requirements 
definition  process. 

In  March  1983,  DOD  established  the  Joint  Executive  Committee  on 
Air  Defense  Threat  Simulators,  commonly  called  the  EXCOM, ^  to 
manage  its  simulator  program.  Among  other  things,  the  EXCOM  is 
supposed  to  review  and  approve  simulator  programs  and  related 
resource  commitments.  Another  committee,  called  the  Crossbow-S, 
is  supposed  to  accomplish  the  coordination  of  simulator  programs 
as  an  agent  of  the  EXCOM. 

We  discussed  the  multiple  developments  in  ERP  programs  with  the 
Chairmen  of  the  EXCOM  and  Crossbow-S.  They  were  aware  that 
multiple  developments  existed  but  considered  it  outside  their 
authority  to  disapprove  service  programs  or  reduce  service 
budgets  to  eliminate  these  efforts. 

In  late  1986,  DOD  prepared  new  proposed  charters  for  the  EXCOM 
and  Crossbow-S  to  more  specifically  define  their  roles  and 
responsibilities.  The  proposed  EXCOM  charter  clearly  identified 


iThe  EXCOM  is  composed  of  repr asentatives  from  the  Office  of  the 
Secretary  of  Defense  [Research  and  Engineering  (Test  and 
Evaluation);  Command,  Control , 'Communication  and  intelligence: 
Operational  Test  and  Evaluation;  and  Strategic  Aeronautical  and 
Theater  Nuclear  Systems]  the  military  services  and  the  Defense 
Intelligence  Agency. 

9 


Page  14 


GAO  NSIAD4J8-93  Electronic  Warfare 


Appendix  I 

Testimony  on  the  Department  of  Defense 
Threat  Simulator  Program 


the  EXCOM  as  the  DOD  decision  authority  for  all  simulator 
programs  with  the  specific  function  of  reviewing  and  approving 
threat  simulator  resources  to  be  included  in  the  Five  Year 
Defense  Plan.  Among  other  things,  the  Crossbow-S  would  be 
responsible  for  reviewing  the  services'  threat  simulator 

requirements  and  resolving  duplications  or  referring  the  issue  to 

I 

the  EXCOM.  However,  the  services  strongly  opposed  the  proposed 
new  charters  because  of  the  authority  which  would  have  been 
vested  in  the  Committees.  This  issue  has  not  been  resolved  to 
date . 

I 

!  Despite  the  uncertainties  about  the  Committees'  authority,  the 

EXCOM  and  Crossbow-S  have  attempted  to  encourage  cooperation 
among  the  services.  In  1984,  the  EXCOM  approved  a  Crossbow-S 

plan  for  the  acquisition  of  various  simulators.  For  each 

i 

simulator  program,  the  plan  designated  a  lead  service  and 
provided  that  other  services  having  a  need  for  the  simulator 
participate  with  the  lead  service  in  a  common  program  to  satisfy 
those  needs.  However,  this  plan  has  not  been  effectively 
enforced  and  has  not  yet  resulted  in  an  effective  joint  program. 

Problems  were  encountered  in  attempting  to  implement  this  plan. 
For  example,  in  its  1984  simulator  program  plan,  the  Crossbow-S 
designated  the  Army  as  the  lead  service  in  acquiring  a  simulator 

i 

for  the  Soviet  SA-1 1 ,  an  advanced  air  defense  system.  The  three 
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services  were  tc  aaree  on  a  connnn  program  to  meet  requirements, 
including  these  for  the  SA-N-7,  the  naval  variant  of  the  SA-11. 
Army  and  Air  Force  officials  signed  the  agreement,  but  Navy 
officials  did  not.  Navy  officials  told  us  that  a  lack  of 
knowledge  about  the  threat  system  caused  deferral  of  the  Navy 
program.  Yet,  the  Navy  has  independently  acquired  a  simulator 
for  the  SA-N-7  and  is  currently  developing  another. 

The  opportunity  for  a  cooperative  joint  program  between  the  Air 
Force  and  Army  also  appears  lost  despite  the  written  agreement. 
First,  the  Air  Force  reallocated  its  SA-11  simulator  funds  to 
another  program.  Then,  the  Army  awarded  a  contract  for  only  the 
Army's  SA-11  simulator  needs.  The  Army  did  not  include  an  option 
in  the  contract  for  Air  Force  needs  because  the  Air  Force  had  not 
informed  the  Army  of  its  needs.  Later,  the  Air  Force  provided 
funds  to  the  Army  to  initiate  its  SA-11  simulator  acquisition, 
but  the  Array  has  not  done  so  because  of  a  lingering  dispute  with 
the  Air  Force  over  the  appropriate  type  of  contract  to  award. 

The  Air  Force  has  insisted  that  the  Army  award  a  fixed-price  type 
contract  while  the  Army  has  maintained  that  a  cost-type  contract 
is  appropriate. 

A  similar  opportunity  for  an  effective  joint  effort  has  been  lost 
in  the  case  of  the  SA-12  simulator  program.  The  Crossbow-S  plan 
designated  the  Air  Force  as  the  lead  service  for  this  program  and 
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provided  that  the  Army  would  participate  with  the  Air  Force  in  a 
joint  effort.  The  Army  then  established  a  low  priority  for  its 
SA-12  simulator  and  made  no  plans  to  acquire  it.  However,  the 
Soviet  SA-12  is  to  replace  the  SA-4  for  which  the  Army  has  a 
simulator.  In  addition,  test  requirements  show  that  the  Army 
needs  a  SA-12  simulator  to  test  several  of  its  major  systems  such 
as  the  Army  tactical  missile  system  and  the  ALQ-136  jammer  which 
is  used  on  the  APACHE  and  COBRA  helicopters.  In  fact,  test 
requirements  for  a  SA-12  simulator  are  almost  as  great  as  those 
for  the  SA-11  simulator  which  the  Army  is  acquiring. 

The  Air  Force  has  attempted  unsuccessfully  to  obtain  Army 
participation  in  a  joint  program  and  is  therefore  acquiring  its 
SA-12  simulator  independently. 

Another  opportunity  has  been  lost  on  the  simulator  progrars  for 
the  SA-10  and  its  naval  variant,  the  SA-N-6.  Even  thouah  these 
systems  appear  to  be  virtually  the  same,  the  Air  Force  is 
developing  a  SA-10  simulator  while  the  Navy  has  a  separate 
program  for  the  SA-N-6  simulator. 

DOD  has  not  established  any  oversight  of  emitter  simulator 
programs.  Representatives  of  the  EXCOM  and  Crossbow-S  told  us 
that  they  had  concentrated  management  efforts  on  the  major  FRF 
simulator  programs  and  were  not  adequately  staffed  to  oversee 
emitter  programs. 
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We  found  widespread  multiple  developments  in  er.it ter  simulator 
programs  within  the  Air  Force.  In  fact,  as  mentioned  previously 
and  shown  on  Table  2,  the  multiple  developments  in  emitter 
programs  are  occurring  almost  exclusively  within  the  Air  Force. 

In  our  opinion,  control  must  be  established  over  the  Air  Force  s 
requirements  definition  process  to  preclude  repetitive 
development  of  simulators  for  the  same  threats. 

I’ll  cite  one  of  several  examples  to  illustrate  my  point.  The 
missions  of  the  Air  Force's  Strategic  Air  Command  are 
characterized  by  long  duration  flights  where  multiple  threats 
could  be  encountered  throughout  the  penetration  run.  To  tram 
its  aircrews,  the  Air  Force  developed  a  simulator  with  the 
capability  to  transmit  numerous  threat  radar  signals  but  whereby 
all  signals  came  from  a  single  point  on  the  training  ranae. 

After  deploying  this  simulator  at  a  cost  of  $61.8  million,  the 
Air  Force  decided  that  it  did  not  realistically  represent  the 
threats  to  the  Strategic  Air  Command  aircrews.  Accordingly,  the 
Air  Force  established  the  requirement  for  another  simulator  which 
would  represent  many  of  the  same  threats  but  which  would  enable 
spreading  the  threat  signals  out  over  the  training  ranae.  The 
Air  Force  is  now  acquiring  this  simulator  at  an  additional 
estimated  cost  of  $196.6  million* 
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MORE  DUPLICATION  POSSIBLE 

We  identified  one  final  but  significant  natter  which  could 
contribute  to  additional  duplication  if  not  carefully  controlled. 
Toward  the  end  of  our  review,  we  learned  that  DOD  has  plans  for 
the  Director  of  Operational  Teat  and  Evaluation  to  begin 
acquiring  simulators.  According  to  the  budget  documents,  the 
"capability  to  conduct  valid  Operational  Test  and  Evaluation 
(OTiE)  does  not  exist  today  for  many  systems."  DOD  has  requested 
$93  million  for  fiscal  year  1988  and  plana  to  request  $138 
million  for  fiscal  year  1989.  With  these  fundB,  the  Director  of 
Operational  Test  and  Evaluation  plans  to  acquire  simulators  for 
the  Soviet  SA-8,  SA-10,  SA-11,  and  SA-12. 

While  we  do  not  disagree  with  the  need  for  these  simulators,  the 
services  already  have  or  are  developing  one  or  more  simulators 
for  each  of  these  threats.  Unless  it  is  adequately  coordinated 
another  simulator  acquisition  activity  added  to  the  existing 
service  acquisition  activities  could  further  proliferate  the 
dupl ication. 
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SIMULATORS  MISREPRESENT 
THREAT  SYSTEMS 

Simulators  being  acquired  are  often  inadequate  for  their  intended 
purpose.  The  simulators  misrepresent  threat  systems  in 
significant  ways  and  thus  adversely  affect  the  testing  of  major 
U . S .  weapon  systems  and  the  training  of  U.S.  forces.  This,  in 
turn,  could  have  potentially  serious  consequences  in  combat. 

We  examined  46  different  simulators,  both  ERPs  and  emitters,  and 
compared  their  characteristics  to  current  intelligence  estimates 
of  the  threat  characteristics.  We  found  that  35  of  the 
simulators  deviated  subs tant ia 1 1 y  from  the  intelligence 
estimates . 

In  cur  evaluation  of  the  simulators,  we  considered  technical 
features  of  the  associated  radars  which  affect  the  system's 
range,  accuracy,  and  resistance  to  countermeasures,  and  thus  the 
overall  effectiveness  of  the  air  defence  system.  These  included 
characteristics  such  as 

--  radar  power  which  affects  the  rarge  at  which  a  radar  can  "see" 
the  target  and  its  capability  to  see  through  jamming; 

--  frequency  agility  which  refers  to  a  radar's  ability  to  change 
frequencies  within  its  operating  frequency  band  and  which 
could  make  the  radar  more  difficult  to  jam; 
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—  radar  beam  size  which  influences  the  system's  ability  to 
detect  targets  as  well  as  the  system's  accuracy;  and 

--  pulse  repetition  frequency  which  refers  to  the  number  of 

pulses  the  radar  generates  per  second  and  which  affects  range, 
accuracy,  and  susceptibility  to  countermeasures. 

We  found  different  types  of  deficiencies  in  comparing  simulator 
characteristics  to  the  threat  characteristics.  For  example,  the 
radar  power  of  one  Air  Force  simulator,  intended  to  test 
electronic  warfare  equipment,  was  only  about  one-third  of  the 
threat's  power  and  was  thus  substantially  less  capable.  Another 
simulator's  ability  to  change  frequencies  rapidly  was  much  less 
than  the  threat's  and  was  therefore  more  vulnerable  to  electronic 
countermeasures . 

Similarly,  an  Army  simulator's  radar  power  was  less  than  one-half 
of  the  threat's  power.  This,  and  other  characteristic 
differences  affecting  its  capability  to  operate  against 
electronic  countermeasures,  rendered  it  misrepresentative  of  the 
real  threat. 

Navy  simulators  also  misrepresented  the  threats.  For  example, 
one  simulator's  radar  had  about  one-third  of  the  threat  radar's 
power,  while  another  simulator  could  not  operate  on  the  same 
frequencies  and  significantly  differed  in  other  features. 
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A  representative  of  DOD ' s  Director  of  Operational  Test  and 
Evaluation  agreed  that  the  simulators  generally  do  not  represent 
the  threat. 

FAULTY  SIMULATORS  COULD  DISTORT 
TEST  RESULTS  OF  MAJOR  SYSTEMS  AND 
TRAINING  EFFECTIVENESS 

Testing  is  a  vital  aspect  of  the  weapons  acquisition  process. 
Testing  enables  evaluation  of  a  weapon's  performance 
effectiveness  to  include  such  issues  as  whether  it  will  defeat  or 
neutralize  specific  threat  systems.  Testing  is  supposed  to 
ensure  DOD  decisionmakers  and  the  Congress  that  defense  systems 
will  perform  as  intended  before  key  decisions  on  their 
acquisi t ion. 

Similarly,  training  military  forces  is  vital.  The  military 
services  strive  tc  train  its  forces  under  conditions  resembling 
combat  as  realistically  as  practical. 

We  found  that  many  weapon  systems  were  being  tested  against 
simulators  that  did  not  adequately  represent  the  threat  and  that 
U.S.  pilots  were  being  trained  against  them. 
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Table  3  lists  some  important  systems  tested  against  ’.re 
s imulators . 


Table  3: 


Systems  Tested  Against 


Deficient 


Simulators 


System 


Mission 


Air  Force: 

ALQ-161 

Jammer 

ALQ-131 

Jammer 

ALQ-135 

Jamme  r 

ALR-56C 

RWR 

ALR-6  9 

RWR 

Self-Protection  for  B-1B  Bomber 
Self-Protection  for  F-16  and  other 
aircraft 

Self-Protection  for  F-15  aircraft 
Threat  Warning  for  F-15  aircraft 
Threat  Warning  for  F-16  aircraft 


Navy  t 

ALR-45F  RWR 
ALR-67  RWR 
ALQ-126B  Jammer 


Threat  warning  for  A-6  and  other 
aircraft 

Threat  warning  for  F-14,  F'A-18 
and  other  aircraft 
Self-Protection  for  F-14,  F  A-1S 
and  other  aircraft 


Army  i 

ALQ-136  Jammer  Self-Protection  for  AH-64 

(APACHE),  and  AH-1  (COBRA) 
Helicopters 

Special  Electronic  Intelligence  and  Electronic 

Mission  Aircraft  Warfare 


We  reviewed  related  test  and  evaluation  reports  to  determine  what 
allowances  were  made  for  differences  between  the  simulators  and 
the  real  threat.  We  found  that  in  most  cases,  the  reports  did 
not  disclose  the  differences.  In  some  cases,  the  reports 
mentioned  simulator  limitations  in  general  but  made  no  attempt  to 
describe  the  deficiencies,  their  significance,  or  the  possible 
effect  on  the  test  results.  The  deficiencies,  if  noted  at  ail, 
appeared  to  have  been  dismissed, in  evaluating  the  weapon's 
potential  effectiveness. 
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For  example,  the  ALO-131  ;anner  :s  provide  se 1 f -protect  ion  for 
several  Air  Force  aircraft,  mcludina  one  of  its  front-line 
fighters,  the  F-16.  To  accomplish  this  function,  the  ALG-131  is 
supposed  to  be  capable  of  deceiving  or  otherwise  interferina  with 
various  radars  used  to  control  SAMs  and  air  iefense  gun  systems. 


In  evalu.tinq  the  ALQ-131's  capability,  the  Air  Force  tested  it 
against  several  simulators  which  were  supposed  to  represent 
specific  Soviet  systems.  However,  each  of  these  simulators 
varied  from  the  related  threat  in  significant  ways.  For  example, 
me  simulator  differed  from,  the  threat  m  power  and  other 
technical  characteristics  to  the  extent  that  it  lii  net  resemble 
the  threat  ir.  trackings  technique  and  a  number  of  ether  functions. 
The  ALQ-13  1  *est  r e per t  did  not  mention  the  simulator 
iefiner.cies  rim  ill  reach  conclusions  about  the  ALC-131's 
effect  iveness . 


Similarly,  the  Navy’s  ALP-6"1  PWP  is  supposed  to  alert  F  A- IB 
aircraft  pilots  that  they  have  beer,  ietected  by  a  radar- 
controlled  weapon.  T*  evaluate  the  ALP-67 • s  operational 
effectiveness,  the  Na  vy  tested  its  per  ft'  rmance  aaams*-  sore 
simulators,  e  i  a  h  t  c  f  wh  i  c  h  varied  substantial!  v  f  r  or  the  r  e  1  a  t  ed 
Soviet  systems  m  radar  power,  pulse  repetition  frequency,  r 
.-her  characteristics. 


Page  21 


<»  AO  \SIAD  K8  fbf  KlerCmnim  Warfare 


Appendix  I 

Testimony  on  the  Department  of  Defense 
Threat  Simulator  Program 


The  test,  report  mentioned  the  simulators  :r.  a  general  way  as  5 
test  limitation  but  did  not  mention  specific  simulator 
deficiencies  or  how  they  might  have  affected  the  ALP-6"  s 
performance.  Yet,  conclusions  were  reached  regarding  the 
system's  potential  effectiveness.  i  erformar.ee  effectiveness 
conclusions  based  on  tests  with  deficient  simulators  could  re 
quite  different  if  the  simulators  had  mere  closely  resembled  the 
threat . 

Deficient  simulators  were  also  being  used  to  train  Y.S.  pilots. 

An  example  would  be  the  Air  Force's  Green  Flac  exercises  held  at 
the  tactical  fighter  weapon  center  ranee.  Green  Flac  is  a 
training  exercise  in  which  tactical  fighter  units  oarticipate  to 
assess  and  improve  their  capability  to  combat  hostile  air  defense 
systems.  Through  the  use  of  threat  simulators,  Green  Flag  is 
supposed  to  provide  an  arena  for  aircrews  to  practice  tactics  m 
a  simulated  combat  environment  and  is  the  Air  Ferre's  main 
electronic  combat  training  exercise. 

We  examined  a  1987  Green  Flag  exercise  m  which  the  Air  Force, 
Army,  Navy,  Marine  Corps,  and  British  Royal  Air  Force 
participated.  Our  purpose  was  to  identify  the  threat  simulators 
used  and  determine  whether  differences  between  the  simulators  and 
real  threat  were  considered.  We  found  that  many  of  the  deficient 
simulators  identified  m  our  review  were  used  in  the  training. 
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Our  review  of  training  reports  showed  that  deficiencies  in  the 
simulators  were  not  considered.  This,  in  our  opinion,  could 
result  in  false  indications  of  force  effectiveness.  For  example, 
one  Air  Force  simulator  varied  substantially  from  the  threat  in 
several  technical  characteristics  such  as  power,  antenna  beam 
width  and  scan  pattern,  signal  processing,  and  the  ability  to 
change  frequencies  rapidly.  These  differences  would  lessen  the 
simulator's  chances  of  detecting  a  target  aircraft  and  would  make 
it  more  susceptible  to  being  effectively  jammed  than  the  real 
threat.  However,  a  pilot  might  successfully  evade  or  neutralize 
the  simulator  whereas  the  results  could  be  different  against  the 
real  threat. 

ABSENCE  OF  EFFECTIVE  INTERNAL  CONTROLS 
CONTRIBUTED  TO  FAULTY  SIMULATORS 

The  services  were  acquiring  faulty  simulators,  at  least  in  part, 
because  of  weak  internal  controls  within  the  services. 

Within  the  Army  and  Navy  there  was  not  a  sufficient  separation  of 
responsibilities  for  the  acquisition,  testing,  and  acceptance  of 
simulators.  Essentially,  a  single  organization  in  each  service 
had  responsibility  for  the  total  process  with  no  effective 
oversight  or  related  checks  and  balances.  Decisions  regarding 
the  acceptability  of  simulators  v th  known  deficiencies  were  made 
by  officials  responsible  for  their  development.  Potential  users 
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of  the  a  um.;  ators,  such  as  the  operational  test  mo  ccr-mur.  if  y , 
were  not  required  to  review  or  approve  such  decisions. 

For  example,  while  developing  a  simulator  to  replicate  the  Sevier 
SA-8  System,  the  Army  Missile  and  Space  Intelligence  Center 
{ MS IC )  learned  that  intelligence  estimates  on  which  the  simulator 
design  was  based  had  changed  significantly.  Rather  than 
initiating  design  changes,  the  MSIC  decided  tc  continue 
developing  the  faulty  design  because  of  tire  and  cost 
considerations.  Subsequently,  MSIC  accepted  two  of  the 
simulators  from  the  contractor  at  S5  million  each  and  furnished 
them  to  the  user  with  known  deficiencies.  The  simulators  were  in. 
use  for  four  years  when  MSIC  decided  to  spend  an  additional  S3.', 
million  to  correct  the  deficiencies  noted  earlier  in  the 
development  program.  The  modifications  have  yet  to  be  validated 
that  they  clearly  resemble  the  threat. 

We  found  a  similar  situation  in  the  Air  Force.  Altheuah  Air 
Force  procedures  provided  for  the  independent  testir.a  -f 
simulators,  the  decisions  regarding  the  adequacy  of  the 
simulators  and  whether  deficiencies  would  be  corrected  rested 
with  the  organization  responsible  for  simulator  development. 

We  believe  the  acquisition  proces3  could  be  strengthened  tv 
establishing  proper  internal  controls  tc  provide  seme  -ess  .re  ; 
independence  to  the  development  end  testma  of  simulators  an; 
related  decisions  regarding  their  acceptability. 
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We  also  observed  that  DOD  was  not  overseeing  the  acquisition 
process  to  assure  adequate  simulator  quality.  In  view  of  the 
cost  and  quality  of  simulators  acquired  by  the  services  and  the 
importance  of  simulators  in  testing  and  training,  such  oversight  I 

seems  desirable. 

i 

ACTIONS  REQUIRED  j 

i 

To  prevent  the  services  from  paying  for  the  development  of  I 

simulators  more  than  once  and  to  assure  that  simulators  acquired 
are  suitable  for  their  intended  purpose,  we  believe  the  Secretary 
of  Defense  should  J 

i 

] 

I 

--  assure  that  the  EXCOM  and  Crossbow-S  or  other  appropriate  DOD 

element  executes  responsibility  and  authority  for  centrally  [ 

I 

managing  simulator  programs  to  provide  for  timely 
identification  and  consolidation  of  simulator  requirements  and 
for  disapproval  of  programs  representing  unwarranted 
development, 

! 

--  require  the  services  to  strengthen  internal  controls  over 

simulator  acquisitions  by  segregating  responsibilities  for  ; 

development,  testing,  and  acceptance  of  simulators  as  valid  j 

representations  of  the  threat,  and 
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—  assign  to  an  appropriate  DOD  element  the  responsibility  for 
monitoring  the  quality  of  simulators  acquired  and 
participating  in  the  acquisition  process  as  necessary  to 
assure  the  adequacy  of  simulators. 

Mr.  Chairman,  this  concludes  my  testimony.  I  would  be  pleased  to 
answer  any  questions  you  or  Members  of  the  Subcommittee  may  have. 
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